Novel Significance Weighting Schemes for Collaborative Filtering: Generating Improved Recommendations in Sparse Environments

Collaborative filtering is the most famous and adopted recommendation algorithm, which recommends items by identifying other similar users, in case of userbased collaborative filtering, or similar items, in case of item-based collaborative filtering. Significance weighting schemes assign different weights to neighboring users/items found against an active user/item. In this paper, we claim that the significance weighting schemes proposed in the literature, are flawed by the fact that they can not be applied to general recommender system datasets. We provide the correct significance weighting schemes using different novel heuristics, and by extensive experimental results on two different datasets, show how significance weighting schemes affect the performance of a recommender system. Furthermore, we claim that the conventional weighted sum prediction formula used in item-based collaborative filtering is not correct for very sparse datasets. We provide the correct prediction formula and empirically evaluate it.